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INTRODUCTION 

It is becoming increasingly clear in the minds of 
those who face the practical problems of improving 
instruction, that t«xt books usually determine the 
success or failure of aoy educational method. And 
yet the various problems involved in text book making 
have been attacked mainly through the hazardous 
and wasteful method of trial uid error, or by the 
method of rough judgments. Recently, however, 
an inclination to regard the selection of text books 
as a matter for scientific study has become prevalent. 
Every study dealing with students or social values or 
economy in learning has been promptly seised upon 
by authors or publishers either to guide the construc- 
tion of new books or to justify books already 
published. 

Studies of social utility in the various school sub- 
jects have been particularly effective in changing the 
content of text books. This is easily seen in the 
reconstruction of books in spelling, and in the reor- 
ganization of the texts in arithmetic. It is now 
practically impossible to sell speUing books which do 
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6 TEXTBOOK SELECTION 

not at least claim to be based upon the recent scientific 
studies <tf the vocabulary of correspondence. 

There is also rapid response to varying types of 
educational theory. Both the subject matter and 
the organization of books are effected as the propa- 
ganda for new theories becomes wide spread. How- 
ever, both publishers and superintendents are coming 
more imd more to distrust mere philosophizing as a 
method of determining the organization of school 
books; tiie publisher because these philosophical 
theories are often transitory; the superintendents 
because books based upon these theories are often 
found to be unsatisfactory. It is becoming increas- 
ii^y clear that all theories for the construction of 
text books must line up with the criteria now available 
from the scientific appraisals of social needs, and with 
the criteria available from the experimental studies in 
the economy of learning. 

These studies in the economy of learning, including 
the study of performance testa, are of the deepest 
significance. Some data are now available in almost 
every field and kind of learning. They range all the 
way from those learning the more elusive problems 
of determining tastes and interests to the somewhat 
simpler problems involved in determinii^ practice 
exercises for specific performances such as hand 
writing. The two studies reported in this book are 
excellent examples of the utilization of scientific 
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TEXTBOOK SELECTION 7 

techniques in detennining student reactions to text 
book materialB. 

At a time when a new emphasis upon motivation 
is being made, Dr. Knight's study of students' choices 
in literature is very timely. We have only begun to 
appreciate the importance of utilizing the method of 
judgments of choices under properly controlled condi- 
tions. Certain aspects of learning are particularly in 
need of just such appraisals. It is fortunate to have 
an example which does not end merely as a statement ' ' 
of a general principle but in a helpful, concrete 
recommendation which can be immediately utilized 
by all teachers of high school Et^lish. 

It was inevitable from the first that the varioua 
studies <^ economy in silent reading would react upon 
the construction of text books. Dr. Franzen's 
investigation was made in the practical emergency of 
text book selection. It illustrates what may be 
expected more and more in the way of substituting 
data for opinion in the solution of the problems 
involved in text book adoptions. There is reason to 
believe that in the next few years we may expect 
much experimental activity aloi^ these lines. 

Both of the studies suggest the interesting possibil- 
ity of subjecting manuscripts to some such experi- 
mentation before printing. It would seem that authors 
owe this to the publishers, and both authors and 
publishers owe this to the public. Of course, scientific 
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8 TEXTBOOK SELECTION 

Studies take time and are expensive. It is not unrea- 
sonable to expect that the profession of teaching is 
now on such a basis that the rewards for taking such 
pains will make the research worth while. 

Ebnest Hobh. 
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CHAPTER ONE 

PRINCIPLES OF TEXT SELECTION 

The major criteria for the selection of text-books 
are five: 

(a) The factor of interest. 

(b) The factor of comprebension. 

(c) The pennaneDt methods of study involved in 
the text. 

id) The permanent value of the content. 

(e) The mechanical construction of the text. 

These criteria possess determinative importance in 
the opinion of educators. There is no one way of 
expressing these criteria. Methods of statement diGFer 
but the meat of the matter is constant. We find 
the above statement convenient for our exposition 
but claim no particular rhetorical superiority for it. 

At present, it is difficult to apply these criteria in 
a particular situation. The application of general 
principles to concrete situations baa always been 
troublesome. Educators find themselves in no uncom- 
mon impasse when they attempt to apply readily 
accepted principles to the specific problems of their 
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12 TEXTBOOK SELECTION 

profesBioQ. The purpose of thia monograph is a 
double one; first to state the fundamental propositions " 
which must underlie sound practice of text-book 
construction; second to give two researches which are "^ 
at least moderately successful attempts at such appli- 
cation. The researches are (a) the application of the 
broad doctrine of interest to the problem of selecting 
texts for high school Ei^lish Literature work, (b) 
the apphcation of the factor of comprehension to the 
selection of geography texts for elementary schools. 

The history of public education in the United States 
is very far from a monotonous report of blunders and 
inadequacies. A study of text-book construction and 
of the methods of text-book selection even now preva- 
lent, however, gives the student of education reason 
to prophesy much increase in the genuine adequacy of 
public school text-books. Further we are sure that 
the selection of a particular text in preference to other 
avaUable books is about to become far less a matter of 
guess and poorly based opinion and far more a matter 
of deliberate choice in the hght of sound criteria of 
text-book excellence. 

Text-book selection is obviously a process of making 
choices, and choices can be made in many ways. It 
is quite possible that certain texts have been chosen 
not because they were better tools of instruction than 
other books, but because the salesman ui^ing their 
adoption had a more persuasive vocabulary, more 
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&^e and plastic sales methods, or was politically 
more canny than his competitors. Other texts have 
had large sales because they were or professed to be 
exponents of some ped^ogical doctrine which momen- 
tarily hypnotized the buying agent. Better than the 
above, many texts have been adopted by schools 
because the proper authority, having studied the 
matter dehberately, chose those particular books. 

Fundamental Propositions 

The thoughtful school administrator will pre- 
sumably subscribe to the following four propositions 
as fundamental to sound practice in text-bookselection. 

I. Irrelevant differences pertainii^ to texts should 
be utterly neglected, viz. pressure from salesmen, 
exhilarating sales tactics. Texts should be chosen in> 
l^t of the five determinative criteria of good text 
construction, namely: 

(a) The factor of interest. ' ' 

(6) The factor of comprehension. 

(c) The permanent methods of study involved in 
the text. 

(d) The permanent value of the content. 

(e) The mechanical construction of the text. 

II. Objective evidence should be given pr^erence 
over subjective opinion wherever objective evidence 
can be reasonably obtained. 
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14 TEXTBOOK SELECTION 

III. Wherever Bubjective opinion is resorted to the 
final judgment should not be the result of the wisdom 
of any one person. Final judgment should be the 
product of the pooled votes of several competent 
judges. Amoi^ these judges should be at least one 
class room teacher who is to use the book. The more 
mutually independent the judgments are the more 
stable the final result. 

lY. Wherever subject opinion has to be substituted 
for objective measurement, especial care should be 
taken to get the best opinion possible. A very good 
judge may sometimes give a very hasty opinion. 

Objective Evidence 

Proposition I, using relevant criteria, and proposition 
III, using the insight of more than one judge, need no 
further discussion. The main part of this book is an 
attempt to apply proposition II to actual, concrete 
school situations. Objective evidence is usu^y 
better than subjective opinion. Many of the decep- 
tions of opinion have been pointed out of late. But 
the objective evidence has its weak points also. Not 
all evidence is good evidence. Mere quantity of 
evidence does not involve in a mysterious way a 
guarantee that the evidence is reliable or consistent 
or determinative. Objectivity of evidence never 
compensates for the errors which come from wrong 
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methods of gathering the evidence or mistakee in the 
interpretation of the ever 8o objective data. 

It is hoped that the two problems of text-book 
selection dealt with in this book wiU be helftful not 
only because of the particular findings but also because 
of the exposition and discussion of the methods used 
to gain objective evidence concerning texts. 

The methods used are of general application. 
Those who are responsible for text-book selection 
should know the reading difficulties not only of geog- 
raphies, but also of all texts, especially arithmetic, 
histories, and science books. 

One of the problems is the determination of the 
reading difficulty of geographies. Obviously a child 
with fifth grade reading ability will work to dis- 
advantage if he uses a text which assumes a reading 
abOity rarely found below the seventh grade. 

The fourth fundamental proposition of text-book 
selection is "Where subjective opinion has to be 
used, care must be taken to obtain the best opinion 
possible." This means the abandoning of general 
opinions and the persistent analysis and estimate 
of the specifics of each text judged. 

We no longer rate a school building bb a whole. 
We judge it for ventilation, for play space, for class 
rooms, for many elements which taken all together 
make a school house. So with text^books, we should 
make our judgments not in general, but in particular. 
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It is apparent that score cards for text-books are useful 
inBtruments to aid in making analyzed judgments. 
Unfortunately such score cards are not in general use. 
In fact for many texts they are not in existence. 

The Use op KATiNa Score Cabdb 

It ia clear that the comparison of one text-book with 
another will be complete when both texts have been 
measured in the light of the five criteria. The whole 
rating of text-books can finally be expressed in mathe- 
matical terms on a score card. For purposes of 
making a decision as to which of several texts to 
adopt the value of each text can, theoretically, be 
expressed in one figure. The figure would be the sum 
of the scores given to each of the five criteria if each 
criterion were considered as important as every other 
criterion. Weighted scores would be needed if the 
criteria were of unequal value. That these five 
criteria are of unequal value seems to be the case. 
From a pool of many judgments we assign following 
point values: 

-^ Interest 200 

Comprehension 250 

Permanent value of subject matter . 250 

Value of method 200 

Mechanical elemeate 100 

Total 1000 
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Score cards or rating schemes of one kind or another 
are generally used to get in orderly fashion expert 
opinion.' Many decisions in practical school affairs 
will have to be based on opinion rather than other 
data for some time to come. Wherever this is the 
case it becomes exceedingly important to obtain the 
best opinion in the best way possible. By best 
opinion we mean the judgment of those not only who 
know most about the matter but the opinion of 
enlightened judges who take the trouble to make the 
best judgment they can make. Here is where the 
score cord is of value. Opinion expressed throi^h 
score card ratings will not give insight the judge 
himself does not possess, but opinion expressed 
through score card ititii^ has a better chance of 
being opinion up to the limit of insight than unanalysed 
opinion has. Thus we see that the worth of opinion 
varies in merit not only with the true wisdom of the 
judge but, also, is a function of the method of obtain- 
ing the opinion. Analysed opinion through score 
card methods increases the chances for a judge to get 
all his goods to market. The better the score card 
the better the judgment will be. A Biology Score 
Card,, one of a series, is given here. Its use makes 
judgment better because it forces the judge to con- 

' OpinioD of snjr degree of expertaesB sbould be resorted te 
only in the abeeoee of evidential data. 
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aider many aspects of hie problem he might otherwise 
neglect. An even better score card would make for 
even better judgments. 
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APPLYING THE CKITEItlON OF INTEREST TO 

HIGH SCHOOL ENGUSH LITBRATURB 

TEXTS 

Suction Ohb 

The busiDeBs of this study is to do two thinga: 

A. Fir8t.~-To name and claasif y in order of inter- 
eat the best liked books commonly need in high 
school literatmB work. This list will be referred to 
as the Preferred List. 

Name in order of interest the next best hked books 
oonunonly used. This list is referred to as the Second 
Preferred List. 

Name in order of interest the third beat liked 
books. This list will be called the Substitute List. 

Name in order of violent dislike books commonly 
used in high schools. This list will be referred to as 
the Free Reading List. 

B. Theseeondseetionof our business is toil) IMemd 
the proposition tiiat the Preferred List of books should 
form the basis or core of the EngUah Uterature work 
in the h^h school, that books in the Second Pre- 
ferred List should be used next frequently, that the 
Substitute List may be drawn from after the Pre- 

22 
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ferred and Second Preferred are exhausted and that 
booka in the Free Reading Liet should never be used.* 

This may be restated as follovB: The best available 
criterion to use in selecting English literature books 
is the amount of interest students have in the books, 
considering only those booka which are now in 
common use. 

(2) Present and interpret data calculated to establish 
the validity of our lists. 

Thb Prefebbed List 

Id the second section of this study sufficiently vahd 
data are presented to warrant the assertion that the 
following are the beat liked books commonly used in 
high schools of lowa.^ 

Careful students who wish to inquire into the factual 
basis of our lists before reading the lists will do better 
if they read the second section first. The busy 
> Books in the Free Reading Hat should never be used as & 
olaae text. These books should be in the librtuy for the use of 
those very few students who wish to lead them. 

' Our data, we hasten to point out, are based on Eenerous 
samplings from high scIiooIb of Iowa. Our criterion of interest 
would apply strictly only to Iowa. The lists of books are genei- 
ally q)plicable in so far as any large group of students have 
tUjout the same interests as large groups of Iowa students. 
Some exceptions in interest ate probably important. For 
instance Vnde Tom't Cabin is highly interesting to Iowa high 
school pupils. Its interest rating would probably be much 
lower in Georgia or Louisiana. 
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school administrator can assume the validity of the 
evidence and content himself with a perusal of the 
lists. 

Table A. — Pbefebbbd List 

RlLIABILTTT 

Intbrest or ' 

Naioi of Book Ratino* RAToni* 

Laddie 97 1.4 

Vicar of Wakefield 92 1.7 

Kenilworth 79 3.7 

Treasure Island 66 2.5 

Hamlet 63 2.5 

Comus 61 3.5 

Uncle Tom's Cabin 61 3.0 

Wild Animals I Have Known 60 5.1 ' 

Ben Hut 59 3.0 

Last Days of Pompeii 59 4.1 

The Spy._ 59 3.6 

Romona 58 4.4 

Kii^ Lear 57 2.2 

The CroBsii^ 55 4.3 

Snowbound 54 2.5 

Luck of Roaring Camp M 4.0 

Macbeth 53 2.2 

Ivanhoe 53 3.8 

Carlyle'a Essay on Bums 53 3.3 

■ Interest rating is in termB of the percentage of etudents, 
who, having read the book, declare that it was of distinct 
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Second PsBrBBRED List 

Incidenta of a French Camp 53 4.2 

The Virginian 52 3.8 

Lea MiserableB 52 4,6 

The Crisis 51 3.1 

The Last of the Mohicans 51 2.6 

Up From Slavery 51 5.5 

Tom Sawyer 50 2.3 

Tale of Two Cities 50 3.0 

Silas Marner 49 2.2 

Man Without a Country 49 2.4 

Huckleberry Finn 49 2.7 

Washington's Farewell Address 49 3.3 

Ancient Mariner 48 2.2 

The Gold Bi^ 48 2.5 

Richard Carvel 48 5.3 

Miles Standish 47 3.2 

Bobinson Crusoe 46 2.6 

Lorna Doone 46 4.0 

intereet to them. Thus the higher the rating, the better the 
book according to our criterioii. 

'Reliability of Rating is to be read as follows: If many 
more estimates of the book were taken, the interest rating might 
go up or down aa much as the reliability figure. The chances 
of its goii^ up or down by on amount greater than tiie reported 
unreliability are very few. Thus Tke Vicar of Wakefi^ has an 
interest rating of 92 and an unreliability of 1.7, which means 
that ae much more data are obtained, the 92 may go up to 93.7 
or down to 90.3. The technique of ascertaining the reliability 
is fully dealt with in section two. 
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Fbxb Kbadinq List 

In Order of Demerit 

CoDtaios books Never to be Used in High School 

Paradise Lost 6 1.4 

Lay of the Last Minstrel 10 2.5 

Heroes and Hero Worship 11 2.8 

Essay on Warren Hastings 12 3.4 

Macauley's Essay on Johnson 12 2.6 

Autocrat of the Breakfast Table 13 2.0 

Vision of Sir Launfal 13 1.2 

Chiide Harold IS 3.5 

Webster's Bunker Hill Speech 19 2.8 

Fairie Queene 20 2.9 

Essays of Elia 21 4.5 

Lays of Ancient Rome 21 4.4 

Utopia 23 4.5 

Thoreau's Walden 23 4.4 

Gareth and Lynette 23 2.9 

The Princess 24 3.5 

Tales of a Wayside Inn 24 2.7 

House of Seven Gables 24 2.1 

Pilgrim's Progress 25 2.5 
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SUBBTITUTB LiST 

Contains in order of probable interest to students 
books leea interesting tban the First and Second Pre- 
ferred List, but of measurably greater interest than the 
books of the Free Reading List. The books are listed 
in order of interest merit. 

Twelfth Night 46 4.9 

Launcelot and Elaine 45 2.6 

Hoosier Schoolmaster 45 3.5 

Christmas Carol 44 2.5 

Hans Brinker 44 3.5 

Legend of Sleepy Hollow 43 2.1 

David Copperfield 43 2.7 

Joan of Arc 43 3.9 

Enoch Arden 43 2.8 

Lincohi's Gettysburg Addrees 42 2.2 

Leatherstockii^ Tales 42 3.6 

Merchant of Venice 41 2.0 

Burke's Conciliation 41 2.9 

Midsummer-Night's Dream 40 3.6 

Prisoner of Chillon 40 3.1 
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The interest ratings of 27 other boolcs frequently 
read in high schools have been determined. The 
First, Second Preferred and Substitute Lista contain 
those of highest ratings. These lists contain more 
books than would ordinarily be used in a four-year 
high school. It seemed needless to print the others. 
These interest ratii^ may be obtained on request. 

Notes on the Lists 

The criterion used in settii^ up an order of merit 
■ of English literature texts is pupils' interest in the 
text. This proposition is defended in Section II. 
Any one whose chmce of texts in the practical situa- 
tion of actual use in high schools is influenced by 
these lists should bear the following in mind: 

(a) The interest which students have in the texts 
is not the only criterion to be used in deciding which 
one of two possible books to use. Other things being 
at all equal, however, it is better psychology and 
sounder common sense to prefer interesting material 
over material of no interest or positive dislike. 

(6) Not every book that could be used in high 
school is included in this study. No book is dealt with 
here unless it has been reported by at least eighty - 
of the high school graduates who rated the list. Thus 
we are concerned only" with books which are already 
fiurly commonly accepted as worth while. This 
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matter could be stated: English literature pro- 
fessore and teachers have on the whole approved of 
many books for high school work. More books are 
generally approved than can possibly be read by any 
one student. Of books accepted as useful which shall 
be chosen? Those of greatest interest to the pupils 
is our thesis. 

(c) It is not certain that the lists are equally sound 
for all sections of the country. In some instances we 
know radical variations in interest would be very 
probable. Children's interests must vary with the 
traditions and interests of the section in which they 
live; how much is not known. Probably books 
vary in their relative universality of appeal. Thus 
Shakespeare may be of about the same interest 
strength everywhere, whereas Uncle Tom's Cabin may 
have high interest in some sections and practically 
none in others. We are well within the facts on the 
other hand to suspect the absence of radical and sharp 
changes in interest. The children of one county 
have about the same interests as those in a contiguous 
county. There is no apparent reason that on a basis 
of children's interests there should be endless variation 
within the same state. It might be added that there 
is probably less difference in interests in city versus 
rural youths than is ordinarily assumed. To differ- 
entiate books on a basis of sex is idle, for boys and 
girls have to read the same books in most high schools. 
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There is no reaeon why our hsts might not very closely 
approximate at least mid-western lists. 

(d) Other criteria should be used in making up a 
list for a particular high school. It is sound to hold 
that other things being equal books in the Preferred 
List should be read sometime during high school. The 
books in excess of this Preferred List should be chosen 
from the Second Preferred List. However, a book 
with an interest rating of 64% poorly taught will prob- 
ably do less service than a book with an interest rating 
of 4S% well taught. Thus, whUe no book should be 
used from the Free Keading List, in a particular situa- 
tion should a teacher much prefer a 48% book to a 
M % book it is good sense to let her teach it, for the 
differential of the teacher's attitude is quite probably 
more important than a small difference in percentage 
of interest. 

(e) Should the preponent elements in the situation 
be a set of slightly lower interest books, on hand, and 
in good condition, a set of slightly higher interest books 
available only through purchase, a supply of funds 
so limited that the purchase of the new books means 
going without some other good thing; then to delay 
for a time the desired replacement is probably entirely 
defensible. 

(f) The interest which one has in any book is due 
not only to the character of the book but also to the 
character of the reader and the method of reading. 
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Thus the same book though read under equivalent 
conditions will not have the same amount of interest 
for all students, not because the book varies but 
because students differ greatly in what they want out 
of a book. Differences may he in many fields such 
as home training, type of play indulged in, amount of 
travel, ability to read, type of fantasies or day dreams 
enjoyed, etc. Thus, even the most interesting books 
on the list can not be expected to interest aU pupils. 
We know they will not. We are pretty sure the books 
in the Preferred List will very likely be actually dis- 
liked by some students. In addition a book which is 
generally preferred to a second one may in some 
pecuhar instances be less Uked by a class or school 
than the second. However the theory of proba- 
bility as used in this study is a sound technique in 
curriculum construction. 

Exceptions need not greatly disttub us. Obviously 
a book in and of itself may be inherently more 
interesting to pupUs than some other. But if the 
other book happens to be well taught and the book 
inherently better happens to be grossly mistaught, 
then the inferior book may very likely appear to be 
better. 

(g) A certain variety in texts is undoubtedly 
desirable. Had the Preferred List on a basis of inter- 
est been all drama, or all poetry, or all heroic novel, 
practical coomderation would force us to draw heavily 
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on other Bchool literature. But the Preferred List as a 
matter of fact is pretty well balanced as far as kinds 
of literary content are concerned. And many wide 
deviations from this list can be comfortably accounted 
for by the consideration that while what large groups 
tend to like, other large groups do not tend to dishke, 
in individual cases no one would expect all pupils to 
like the same books and in the same relative order of 
interest. These hsts do not pretend to be correct 
for all students, but as far as we now know they will 
strong tend to be better than any other list. 

(h) It ia not useful to think of the most interesting 
book as the "beat" book or the least interesting book 
as the "worst" book. The most interesting book is 
Laddie. No one would claim that Laddie is the best 
book in the sense that "if only one book was to be 
read in high school this book should be read." Such 
choices and Buch "ifs" do not have to be made. 
Many books are read. The question is which group is 
best. In the best group Laddie finds a place and 
Paradiae Lost does not. This placing is, of course, 
accordii^ to oiu* criterion of interest.. 
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Reapbrs' Intebkbt the Cbitbbion^ 

The criterion used in the construction of these lists 
- is the interest of the student as reported. 

No one aware of the adminlBtrative exigencies of a 
high school would contend that "what pupils like" 
^ould be the sole criterion for admittli^ a book, a 
subject, a technique, or a rule into the total economy 
of high school procedure. Neither would one suspect 
that what pupils dislike should be the sole criterion 
for dropping & book, a subject, a technique, or a rule 
from tiie school. 

It is hardly conceivable that any teaclier is so 
unenlightened as to hold that disagreeable tasks are 
worth while because of their disagreeableness. A 
task may be distasteful and yet worth while, but its 
worth-while-ness is certainly due to other factors than 
its unpleasing nature. 

Theoretically, most competent English teachers 
would agree that one of the criteria safely used to 
select texts would be students' interest, but there 
would be little agreement on the amount of influence 
interest should have. If the books available should 

' Hieoretical coiuideration of the importtuioe of interest is 
asBumed. Petaonailj, we view those who do not appreciate 
the pragmatic importance of mterest in education as in need of 
inatiuotion in the simple principlea of the teaching process. 

Nor are we concerned here with the objective of our English 
Idtetature work. 
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be scored on a basis of 100 points, would literary 
merit be assigned 50 points, moral teaching 25, and 
interest 25, or would it be reading difficulty of the 
composition 50, literary merit 15, interest 15, amount 
book is used in other Bchools 20, — or what relative 
importance would be assigned to interest? 

In our present ignorance of the details of scientific 
management in education, one would utterly despair 
of getting definite answers to questions of this kind. 
However answers to such questions must be obtained 
before we can speak of education as a science in the 
sense that civil engineering is a science. All would 
agree ih&t interest is only one of several factors which 
properly influence selection. How influential a factor 
it should be has not yet been scientifically established. 
Our data show that the seUdion of English literature 
in the State of Imoa praOieally reuses to recognize the 
- factor of intereet at all. The frequency with which 
' books are read in Iowa high schools and the frequency 
with which students report these books as interesting 
to them have practically no correlation other than 
that of chance. This roughly may be taken to mean 
that the more frequently read books are rated as less 
than average interest just as often as they are rated 
as above average interest, also that ^e less frequently 
read books receive an interest rating greater than the 
average as often as "lees than the average" rating. 
Such a state of afFurs would be jtMifiable if it could 
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be clearly ehovn that other crit«ria more important 
than students' interest determined the frequency with 
which books are read.' That such criteria do not 
operate will be shown and evidence given later in this 
study. 
— AMumin^ that the 783 reports on which this study 
is based are a fair sampling, the following examples of 
the strained relations if not absolute divorce between 
"how often books are read" and "how much books 
are liked " will be illuminating. 

The 10 books read with the greatest interest were 
• recorded on the average 280 times. 

The 20 books read with enoi^h interest to warrant 
being in a Second Preferred or First Substitute Lists 
were read on the average 247 times. 

The 20 books read with so little interest that they 
should not be read at all were read on the average 
203 times. 

The average interest rating of the Preferred List 
is 62.7%. 

The average interest rating of the Free reading List 
is 16.3%. 

Thus, while in frequency the shrinkage in use 

' between the best and worst lists is only 17.6%, the 

' shrinkage in interest is 46.4%I The shrinkage in 

interest is approximately 2.6 times as great as the 

ahrinkage in frequency of use between the I^referred 

and the Free Reading List. I doubt if any one would 
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want to defend this relationship. It certainly would 
never have happened if the choosers of text-books 
had known the interest in books before makii^ their 
choices. 

To those who may suspect that a study of only 
the best books and the worst in point of interest bides 
a more favorable state of affairs because it does not 
consider the middle books at all, the relation between 
frequency of use and amount of interest of aU books 
will be given. 

The technique here is that of correlation. 

We digress to explain what is meant by correlation. 
Persons f amiUar with statistical theory should skip the 
next page. 

The coefficient of correlation is a measure of con- 
comitant variation between two series of facts about 
the same data. The limits of this coefficient are 
— 1 and +1. If the relationship is +1 it means that 
there is a perfect matching. Thus if there were a -HI 
correlation between the frequency with which the 
books we are studying are read and the amount of 
interests pupils have in them, then the following 
conditions would hold : 

The book read most frequently would also be the 
most interestii^; the book read least frequently would 
also be the least interesting; the book read with the 
fifth greatest frequency would also be the fifth book in 
interest; ihe book read next to the least often would 
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be the book next to the bottom in intereet, and 
80 on. 

If there were a — 1 correlation between frequency 
and interest then just the reverse would be true, i.e., 
the most read book would be the least interesting; the 
book fifth from the top in frequency of the reading 
would be fifth from the bottom in its interest rating. 

If the correlation were zero then there would be 
no relationship but a chance one between frequency 
and interest. Some much read books would be high 
in interest, others low, others average. Some books 
low in frequency would be low in interest, others high, 
others average in interest. Thus the sign and size of 
the correlation coefficient measures the amount of 
mutual relationship between the frequency with which 
books in our list are read and the amount of interest 
pupils have in these same books.^ 

Using only the books which were reported eighty 
times or more (the unreUability ot ratings based on less 
than eighty gets too large for conservative study) we 
find the correlation between frequency of use and 
interest to be — .04, practically sero. 

■ Further explanation of the correlation ooeffieient may be 

Thomdike. Mental and Soeiat Meantrementt. 
Brown and Thomson. Euenlude of Mental Meaawement. 
Ragg. StcUUHcdl Method Applied to EdveaUon. 
Refer to index in each book. 
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This means that there is Qo tendency to read the 
more intereBting books oftener than the non-interest- 
ing one. There is also no tendency to read the non- 
interesting ones any more frequently than the 
interesting ones. 

In view of the fact tiiat the data give also a rating of 
positive dislike it is instructive to study the correlation 
between the frequency with which books are read 
and the positive dislike for them. It should be 
pointed out that the positive dislike ratings are 
entirely separate from any interest ratings. They 
were made independently. The correlation is —.52 ' 
between frequency of reading and positive dislike. 
This means that there is a pretty definite factor of 
negative selection going on at present. The nearest 
approach to anything like scientific control that 
the data reveal is this fact that on the whole books 
which are positively disliked tend to be read less often 
and there is a slight tendency to read them less often ' 
in the order of the strenuousness of dislike . 

Certainly the factor of interest should operate in the 
selection of tests. But it does not. We are not 
prepared to say that it is theoretically impoaeible to do 
good work in a subject in which the learner has no 
interest, but interest is a factor which facilitatea 
learning. On the whole, interest in the work runs 
along with skill in it. With material such as litera- 
ture the interest factors are probably more important 
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than in more mechanical subjects where physical 
skill based oo habit predominates. If reading books 
carries no interest but often even positive annoyance, 
there is little if any likelihood that reading goes on 
except under compulsion. The frequent task of 
reading inherently uninteresting material under school 
compulsion may account for whatever truth there 
is in the general feeling that adults on the whole 
read littie hterature of great merit. Very few high 
school graduates ever read the ^neid over again for 
the fun of it. We doubt if one high school graduate in 
fifty, who, having read Sesame and LUiea or Paradise 
Lost in high school, ever reads these books ^ain, or 
reads other works by the same authors unless they are 
in turn teaching them in high school for a wage, to 
pupils who will never look at them or think of them 
again unless later as high school teachers, they in 
turn — and so on.' 

Several cross sections of the data are presented here 
to further illustrate the divorce between interest and 
frequency of reading certain books. 

Table B contains the 29 books read the most fre- 
quently. On these we have 350 reporte or more. ' 

' Hie way a book is taught muot determine in put the inter- 
est pupils have in it. This relation between how a book is 
taught and resulting inteiest is as yet unknown. 
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No. OF 

Taos PlACB ON iNTZBaST 

Refobted Nakx or Book Ratino' 

720 Vision of Sir Launf al Free Reading List 

607 Merchant of Venice Substitute 

552 Legend of Sleepy Hollow Substitute 

535 Lady of the Lake. No place 

529 Silas Marner Second Preferred 

523 Macbeth First Preferred 

523 Liacoln's Gettysburg Speech Substitute 

503 Ancient Mariner Second Preferred 

492 Ivanhoe First Preferred 

481 Courtship of Miles Standish. Second Preferred 

465 Great Stone Face No place 

460 The Raven No place 

459 Julius Caesar No place 

424 Man Without a Country Second Preferred 

414 Gold Bug Second Preferred 

411 House of Seven Gables Free Reading List 

397 Christmas Carol Substitute 

385 Last of the Mohicans Second Preferred 

383 Canterbury Tales No place 

383 Snowbound First Preferred 

380 Passing of Arthur No place 

378 Sketch Book No place 

375 Hamlet First Preferred 

369 Treasure Island First Preferred 

368 Thanatopsis No place 

368 Idylls of the King No place 

366 Robinson Crusoe Second Preferred 

366 Gray's Elegy No place 

357 Launcelot and Elaine Substitute 

> "No place" meaoB that Uie book has an interest ratiDg 
bettet than the Fine Keading Idst but not high enough to leaeh 
even oui Substitute Uat. 
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Of the 29 most frequently read books (350 reports 
or more out of 783 students) 

5 are in our First Preferred List 
7 are in our Second Preferred List 
5 are in our Substitute List 
10 are in "Not Placed" list 
2 are in the Free Reading List 
Total 29 

Of the ten booka^which had an interest rating 
of 45, one was read 640 times and one but 160 times. 
The other eight were rather evenly distributed 
between these extremes. Of the twelve books read 
with an equal frequency (280 times) one had an inter- 
est rating of 10,' another of 95. And here the other 
ten were scattered between these extremes. 

All the data show how books with equal interest 
are read with h^hly variable frequency. Books read 
with equal frequency have highly variable interest- 
value. 

So far we have shown that interest as such exerts 
no measurable influence on frequency of reading over 
the state of Iowa as a whole. We wish to present 
data to show that the operation of other definite 
criteria is not evident. 

If clean cut criteria were operating under conscious 
direction some one would know about it. Who does? 
What are the criteria? Reading difficulty is a very 
good criterion, but the reading difficulty of books 
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used Id high school English literature has never been 
determined. Some one could very well determine this. 
Perhaps information value is thought by some to 
be an effective criterion which we have carelessly over- 
looked. We have yet to see a study of the information 
value of the books under consideration. Such a 
study might well be made. At present, however, this 
criterion is still a "poor relative" of opinion and a 
very much be-fussed relative into the bai^ain. It 
would be difficult to show that the much read books 
are higher in information value than the much liked 
books. Comparing the First Preferred List with the 
Free Reading last one might perhaps be impressed 
with the realism of the Preferred List in contrast to 
the fantasy of the Free Reading List. Possibly 
people have to be older than of high school age before 
they become sufficiently disillusioned about the world 
to find pleasure in fantasy. The h%h school pupil 
may be still satisfied with the world as it is, or nearly 
is. He may not yet be interested in Utopias, Farar 
dises Lost or Found, Fairie Queens or Visions, or he 
may be too old to be interested in them. 

LITSaAJBT UBSIT NO CRITBBIDN 

It is always possible to fall back on the well known 
defense of Uterary merit. It could be asserted, if 
one is careless enough about the facte, that the 
criterion of literary merit decides the frequency with 
which books on the whole are read in high school. 
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In one sense this is true. Probably no book is much 
read until by a vague consensus of opinion it is worth 
reading. But this does not effect our data because 
we have considered only books rather frequently 
read. They all possess enough literary merit to be 
generally used. But differences in literary merit in 
excess (rf a minimum amount of it to allow entrance 
of books into common use can not account for the 
frequencies. 

We have accepted literary merit in this sense, as 
a working criterion of a minimum amount. It is 
important to realize that in no other sense can we hold 
it to be influential. It is not at present a criterion for 
two reasons. First, there is no objective measure to 
use as a criterion. There may be some day a scale for 
rating for literary merit. Up to now contributions on 
the matter have been much cluttered with opinions 
and not disciplined with scientific or orderly study. 
In short, every one has a literary rating all bis own, 
(which rating we suspect is largely a rationalization of 
likes and dislikes by the individual doing the rating). 

Second. — If differences in literary merit decided 
differences of frequencies of use so that the most meri- 
torious book was read the most frequently, the second 
most meritorious book was read the next most fre- 
quently, and so on, then there would have to be a 
general agreement as to the differences in literary merit 
of the books concerned. There is not even an approxi- 
mation of agreement, as the following data show : 
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We have had five groups of judges rate the nineteen 
books composing the Preferred List and the nineteen 
booka composing the Free Reading List for the quality, 
literary merit. The judges were in all instances igno* 
^rant of the interest value of these books.. They were 
ui^d to give their own opinion independent (as far as 
possible) of fame of the author, popular appeal of the 
book, etc. 

The value A was to be given to those books which 
were of preeminent literary value, the great books 
of the ages. 

The value E was to be given to those books which 
were of such poor quality that probably they should 
not be considered for serious work at all. 

The value C was to be given to books about half way 
between A and E. 

The value B was to be assigned to books better 
than C but of less merit than A. 

The value D was to be assigned to books worse than 
C but better than E. 

The additional instruction was given that in case 
of unfamiliarity with a book no rating need be given. 

The pooled results of these judgments of the literary 
value of the Preferred List (each book given a number) 
and of the Free Reading List (each book given a 
number) is presented. The reader is asked to dis- 
tii^uisb on a basis of literary merit which list of 
books is the best. 
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List X' 
Showii^ the distributioD of judgmenta 
books for literary merit. A, B, C, D, E, 
merit as defined in text. Kumbers 1, 2, 
to books. 



on nineteen 
amounts of 
3, etc. refer 





A 


B 


C 


D 


E 
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3 


21 


27 
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3 


60 


2 


9 


41 


30 


6 


1 


87 


3 


21 


27 


7 


2 
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4 


16 


56 


43 


4 
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5 


62 


46 


16 


11 


2 
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3 


34 


22 


10 
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25 


30 


38 


22 
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32 


10 


15 


10 


91 
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61 


31 


36 
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122 
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14 
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49 
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33 


38 


13 
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22 


26 


39 


13 
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13 


48 


31 


26 
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14 


14 


22 


21 
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81 


15 


30 


33 


47 


7 
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16 


9 


31 


20 


12 




75 


17 


53 


37 


16 


6 




113 


18 


18 


61 


29 


4 




103 


19 


7 


17 


21 


12 


6 


63 



428 



175 



1817 



627 535 

' Whether this list is the Pref ened or the Free Reading list 
will be stated later on. This poatponement is to enable the 
nader to aee the utter hopeleeBness of getting a "literary 
merit" when the books are given by number rather than name. 
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The reader will readily see that a great amount 
of sagacity would be necessary to determine whether 
List X or Ijst Y is superior in literary merit. There' ia 
certainly not enough difference to outweigh the essen- 
tial difference in^interest value which exists between 
the two lists. 

The careful student will immediately wonder if our 
pooled ratings are not a mixture of inexpert and con- 
sequently valueless opinion. We shall therefore spread 
out the pith of each group of ratings to show that 
there is no difference of importance between the 
judgment of groups which vary in expertness. 

Group I is a class in psychology of advertising. 
This group is not expert in matters of literary merit of 
English literature. Their judgments are: 

List X List Y 

of 369 judgments of 230 judgments 

on 19 texts on 19 texts 

21.4% judgments were A 12.2% were A 

35.6% judgments were B 38.7% were B 

31.5% judgments were C 42.2% wore C 

9.5% judgments were D 4.3% were D 

1.9% judgments were E 2.6% were E 

We interpret these data as showing that in the 
opinion of a psychology of advertising class List X 
is slightly superior to List Y in literary merit. 
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Group B is a class of collie juniors preparing to 
teach in high schools. We aaeume that their judg- 
ments are slightly more expert than the judgmentB of 
Group A. 



List X 

of 995 judgments 
OD 19 texts 

28.3% of judgments were A 
36.8% of judgmentB were B 
24.7% of judgments were C 
6.6% of judgments were D 
3.5% of judgdients were E 



List Y 

of 803 judgments 
on 19 texts 

19.9% were A 
39.9% were B 
29.0% were C 
8.5% were D 
3.6% were E 



In the judgment of Group B List X appears to be 
slightly better than List Y. The chief characteristic 
of this difference is however its negligible smallness. 
Other criteria than literary merit should obviously 
operate in deciding which list to use. 

Group C is a graduate seminar in educational 
psychology. Most of the group are experienced 
teachers. They are presumably still more expert 
than either Group A or B. 
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List X List Y 

of 216 judgmente of 222 judgments 

on 19 texts on 19 teste 

26.3% of judgments were A 36.5% were A 

35.6% of judgmente were B 31.1 % were B 

27.3% of judgmente were C 23.9% were C 

9.3% of judgmente were D 6.3% were D 

1.4% of judgmente were E 2.3% were E 

Here List X has 61.9% of judgments B and A 
List Y has 67.6% of judgments B and A. 

There is a slight difference in total literary merit in 
favor of List Y. But the difference is so slight that no 
one could possibly judge it to be as important psycho- 
logically, pedagogically, or culturally from the teach- 
ing standpoint as is the difference in interest value 
determined by this study. 

Group D, our most expert group, is Doctor 
Craig's seminar in introduction to graduate work in 
English. Many of the group are instructors or have 
been instructors in English literature. They have 
profited by unquestioned expertness in English 
literature through graduate study in English. 
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List X List Y 

of 182 judgments of 176 judgmeota 

on 19 texts on 19 texts 

22.5% of judgmente were A 14.2% were A 

21.9% of judgments were B 27.8% were B 

34.6% of judgments were C 38.6% were C 

15.9% of judgments were D 14.7% were D 

4.8% of judgments were E 4.5% were E 

Two facie are apparent: (1) the expert group rated 
both lists on the whole lower than the relatively less 
expert groups rated them; (2) in the judgment of the 
most expert group List X and List Y are practically of 
equal hterary merit. Perhaps List X is slightly 
better. In all cases hist X is our Preferred List and Y 
our Free Beading List. 

The force of the argument is now apparently 
adequate. 
When groups of judges varying in expertness are 
, asked to pass on two lists of books the experts hand in 
about the same findii^s as the less expert. There is 
then no expert opinion as different from common 
knowledge. For if there were a useful expertness 
graduate students in hterature would have it. They 
do not. Our blunt conclusion is: since judgments of 
literary merit on our Preferred List as a whole and on 
our Free Reading List as a whole can not be made to 
even suggest significant differences, then literary- 
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' merit as a criterion, except as we have already admitted 
it' is not determinative. Thia we take it leaves a clear 
field for the validity and uaefuloess of the lists pre- 
sented in the first of this study which are based on the 
criterion of interest.* 

This lack of agreement in matters of merit is not 
surprising. After all literary merit or any of its 
synonyms contains but little useful truth. It is 
similar to such phrases as the Well-rounded Life, 
Education for Complete Living, True Culture, Good 
Citizenship, and the like. We assume that any phrase 
such as those contains a notion upon which many can 
agree as they can agree on the notion that two plus 
two equals four. No such notion is contained by such 
a phrase as Literary Merit, or Education for Complete 
Living, or Good Citizenship. These are verbal fog 
screens concealing chiefly differences of opinion. 
After all what may appear to you as a well-rounded life 

■ Minimum amount to be admitted at aJl. 

' If the Teader should feel that there are a few books in the 
Free Reading List of such great merit that they should be 
read in spite of their low interest value he can eatisfy himaelf 
as to the difference of opinion on the matter by attempting to 
pick out from the pooled judgments of List Y which booka Ihty 
are hy the spread of judgments presented then. He might 
poBsibly pick number 6, 13, 17, which are AiOocrat of the 
Breakifast Table, Utopia, Talea ^ Wayside Inn. No one of 
these books are really liked by one pupil out of four who have 
read them. 
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may appear to your neighbor as the life of a prig, a 
selfifih hoarder, a silly person, or what not. While hie 
notion of a well-rounded life may seem to you to be a 
life too physical, too intellectual, too ethereal, or too 
worldly. And all the while we know you are both 
wrong and that our notion of the well-rounded life 
though sharply difFering from both of yours is really 
correct. 

So with literary merit, perhaps that is meritorious 
which is liked or which satisfies, or in some other way 
leaves a characteristic impression on the reader. As 
you and your neighbor are forever satisfied with dif- 
ferent things, literary merit is one thing to you and 
another to him. 

Even though this explanation is faulty the cold 
fact remains that there is as yet too little agreement 
as to the varying worth of the books we are considering 
to use merit as a criterion. 

Third, — if this were at present a criterion, it cer- 
tainly is not effective. For who would say the follow- 
ing differences in frequency of reading were due to 
clear differences in literary value : 
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ViBion of Sir 

Launfal read 720 times out of 783 reports 

King Lear read 185 times out of 783 reports 

Merchant of Venice . read 607 times out of 783 reports 

Hamlet read 375 times out of 783 reports 

Julius Caesar. .'. read 459 times out of 783 reports 

Treasure Island read 369 times out of 783 reports 

Sesame and IJlies. . .read 145 times out of 783 reports 
Gold Bug read 414 times out of 783 reports 

Another defense for the present situation would be 
that a balanced literary ration must be provided for 
high school pupils and the present practices are 
approximations of this balance. The necessity for 
balance is of course entirely proper and sound. But 
there is some difficulty in justifying the wide spread 
notions about just what a balanced ration is. How- 
ever, it is to be noted that there is at least variety in 
the Preferred List as well as in the other lists. There 
is a fairly good variety both as to the type of literature, 
date of publications (i.e., age) and many of the books 
in our lists have undisputed literary merit. We 
doubt if in the matter of balance the books most often 
read make any better presentation than do the books 
which when read are of distinct interest. 

The present situation in English literature as far 
as the frequency with which books are read is con- 
cerned, is certainly not the result of conscious, 
purposeful, scientific control and direction, right or 
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wrong. It is a result of s confusion of influence hope- 
lessly interwoven. In addition it should be kept 
clearly in mind that even if other criteria were oper- 
ating still the present impotence of the interest factor 
can be viewed with philosophic calm only by the 
pedant.* 

The evidence so far supports the following 
statements : 

The correlation between ihe criterion of interest 
and the frequency of reading is zero. 

The criterion of reading difficulty can not operate 
in a useful fashion for the difTerenee in reading diffi- 
culty has never been determined. 

The criterion of useful information is not operative 
for the useful information rating of the books is too 
nebulous. 

Literary merit doubtless operates in the restricted 
sense of determining in some way the entrance of 
a book into use at all. But it does not determine the 
relative importance of books that are used, for there 

* It can Im ahown BtatiBticaUy that the operiitioii of deSnite 
criteria need oot be aBSumed to account for the present situa- 
tion. For the frequencjr with which books are read confona 
to the upper half of a pure chance curve within the probable 
error. The treatment of the data to prove this b deleted for we 
do not need it to prove our point concerning the absence of 
steeling and the prevalence of drift at present. Moreover our 
interest is not to prove things are wrong but to sell l>etter 
EngUsh literature choices tt> school managers. 
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is not sufficient agreement as to the measuring of 
literary merit. 

We repeat that the use of some criterion though 
imperfect is better than the vague and uncertain 
practices which now exist. If choices of high school 
literature texts were made in the light of the hsla 
of this study the situation would not be made worse 
and would probably be genuinely improved. The 
criterion of interest is not only known but is of 
respectable importance in any teaching situation. 

Section II 

We have presented lists of Uterature texts. 

We have defended the use of the criterion of interest 
in setting up lists. 

However, the lists are good only to the extent 
that the data upon which they are based are valid. 
We now present a description of the original data. 

The freshmen EngUsh classes at the State University - 
of Iowa were given as an exercise the filling out of the 
following blank.* 

We took freahmEin data not beoause those only were avail- 
able, but because freehmen t^peaied to ua to be the proper 
judges. High school senion are a little too near Bome books to 

I Tbt data were obtained through the Engliah Department 
of the State University of Iowa of whioh ProfeeBoi H. C. Cnug 
is the head. For this courtesy we desire to express our appre- 
ciation. 
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hare » good penpective. College seniors' ratings would have 
been the result of origjn&l preference plus four yean of coDege 
training which would certainly tend to change their attitude 
toward some books. Seniors would tend to rate not on what 
they did like but on what they now think they ought to like. 
There would be a repetition of the "old oaken bucket" delu- 
sion. The necessity of carrying water for mother, the weight 
of well sweep, the leaks in the bucket, the spilled water, the ice 
on the well, all tend to be forgotten as distance softens tiie hard 
spots and enchants the pleasantness of childhood. I suspect 
that many honest grown-upa really think that in their youth 
they said at story time " Mother, tell us a story and be sure to 
put in a wholesome moral." 
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1. Check ID the first column those books which you studied in 
class durkig youi high school course. 

2. Check in the second column those books which you read out 
of claw «s collateral reading. 

3. Check in the third column those books which you now believe 
to have been of the most interest. 

4. Check in the fourth column those books which you now 
believe to have been of no particular interest, and yet not 
entirely lacking of interest. 

fi. Check in the fifth column those books which you now believe 
to have been of no interest whatever, and worthless. 

6. Check in the sixth column those books which you would drop 
from a course. 

7. At the bottran of the last page you will find apace. List here 
a number of books you would add to a course. 

Name 

High school last attended 

Major study in 8. U. I 



Burke's Conciliation of the Colonies. 

McCaulay's Elesay on Ben Johnson. 

Macbeth. 

King Lear. 

Midsummer-Night's Dream. 

Twelfth Night. 

Hamlet. 

The Ancient Mariner. 

Silas Mamer. 

Ivanhoe. 

Vicar of Wakefield. 

Snowboimd. 

Roger De Coverly Papers. 

living's Sketch Book. 

Great Stone Face. 

The Laat of the Mohicans. 

He Vision of Sir Launf lU. 

The Courtship of Miles Standish. 



6 6 
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This list contains 108 books. 

Returns were received from 732 students. These 
data were tabulated in ways to permit of convenient 
study. 

The correlations which are reported in this study 
were computed by the Pearson formula 
Say' 

There are, of course, certain objections which can 
be raised about the tnistwortfainess of the data. I 
shall consider first the trustworthiness of the data aa 
to the quantity, then as to the quality of it. 

QUANTITT OP THE DATA 

The ratings of the books in this study can be 
accepted with confidence as far as the amount of evid- 
ence is concerned. We shall discuss at some length 
the reliability of the ratings because they are expressed 
in percentages. The reliability of percentages has so 
far been too little considered in scientific studies in 
education. 732 questionnaires were returned. Not 
all the books were read by every one of the 732 
students. In fact, no book was reported on by all. 
Obviously the more cases there are in any study the 
greater the reliabiUty of the findings, other things 

* See Thorodike Menial and Social MeaauTements, page 173. 
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being at alt equal. We could have wished to base our 
interest ratii^ on 7320 cases instead of 732. Our 
evidence is much stronger because it is based on 732 
reports than it would have been if only 73 students 
bad rated the books. 

Since the reliability of the ratings is a function 
of the number of reports studied we want to know 
just how reliable our findings are. To illustrate: 
HavUet was judged by 375 students. 63% judged 
the book to be of genuine interest to them. Can we 
generalize from this and say 63 % of high school pupils 
taken at large find Hamlet of distinct appeal to them? 
Of course the ideal way to determine what percentage 
of h^h school pupils find Hamlet of especial fasci- 
nation would be to aak all of those who had read 
Hamiet. 

Less ideal than this way but far better than guessing 
is the determination of the reliability of the 63%. 
This can be done by the application of the formula of 
simple sampling. Each one of the 375 students, who 
having read Hamlet, rated it, can be thought of for 
our purposes as performing a little experiment. He 
is to rate Hamlet as interesting or not interesting. 
376 rate HanU^ and 63% of them rate it as 
interesting. Suppose we went on and got 700 ratings. 
Would 63 % of the 700 rate as did the first 375? The 
percentage of 63 would probably vary slightly. But 
variations would probably center around 63. After 
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700 or 1,700 more studenta had been asked we should 
expect their percentage of interest to be nearer 63 
than 12 or 15 or even 50 or 90. That is, on our 375 
cases we should expect that the interest ratings of a 
much larger group would not be exactly 63 but that 
63 would be the best single bet we could make in the 
I^t of our iacts. 

The formula '\/npq when n = number of cases, p = % 
of pupils who rate Hamlet as interestii^, and q the % 
who rated the book as not interesting, will tell ua the 
standard measiue of variation from 63 which the 
percentage of interest will take as more casea are 
added. 

To illustrate the process in detul : 

n = 375 students rated Hamlet. 

p = 63% of these rated Hamlet as interesting. 

q = 37% of these did not rate Hamiet as interesting. 

With 375 cases to predict from how will the per- 
centage of interest probably vary as many more casea 
are added? 



This is found by applying formula - 
In our illustration ^^ = 2.55%. 



> The qnantitf y/npq ia iteelf divided by n to cbange H bom 
a quantity in tenna of n to a percentage. 
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This meauB that as more cases are added 63% may 
go up 2.55% or down 2.55%. The probability that 
the % of interest will not go over 65.55% (63 + 2.55) 
or below 61.45% {63 - 2.55) are 68 chances out of 
100. The probability of the interest in Hamlet being 
less than 57 or more than 69 is but two chances out of 
100. It is evident that, when only those cases where 
n is 80 or more are involved, the reliability of our 
findings is very high indeed. So much for the 
validity of our data as to quantity.* 

It is possible that we have enough data but its 
quality is such that it is untrustworthy. Here the 
argument would run as follows. If you ask any num- 
ber of people a question about which they know 
nothing the answers are no better than guesses. The 
average of many fake answers is not a right answer 
except by chance. 

College freshmen may not know why they liked 
and disliked certain books in high school, but it is 
reasonable to assume the plain facte that pupils do 
like and dislike books and that they know what 
interests them and what does not. If their reports 
are false they are false not because knowledge was 



■ Those who wish to further atttiHfy themselves on either the 
BOirndDeae of the theoty of "Sampling" or on the technique 
involved will find Yule'a IntrodueUon to the Theory of Statutia, 
CiuepUir XIII, a satisfactory reference. 
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laeking but because they purposely lied or reported 
vitb hopeless careiessness. 

There was certainly no reason for deception. There 
may have been a slight tendency to say the proper 
thing which in this case would be to report more 
favorably than their true interests warrant on books 
which are famous, or old, or "high brow" and to 
underrate their biie interest in the so-called popular 
books. The fact that the percentages of positively 
disliked books are relatively small may perhaps be 
interpreted as a result of two factors: — genuine 
dislike minus a hesitation of going on record as 
dislikii^ books that one "ought" to like. To the 
extent that our data are vitiated by this hesitation, 
it gives a case weaker not stronger than the true 
case in favor of our Preferred List and ^unst the 
Free Reading list. 

The foUowii^ rdationships (A the data are our 
evidence of the validity of the data. 

The correlation between interest and a neutral 
attitude toward the books was — .18. 

The correlation between neutral attitude and dislike 
waa +-5I. 

The correlation between dislikes and dislike so 
great that dropping the book in question was recom- 
mended waa +.67. 

These tiuee correlations go right against any notion 
that the ratings were the product of carelessness. In 
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fact, if the markings were the product of carelesaneaa 
rather than of thought we simply could not accoxmt 
for these correlations. 

The correlation between interest and a neutral 
attitude (—.18) is practically no relationship at all 
other than a chance one. And it is about what one 
could reasonably expect to be the fact. The great 
interest that some have in a book can very well run 
tdong with a neutral, 50-50, attitude of others. 
Unless one is under the spell of the "pathetic fallacy" 
he would not expect to be able to establish any high 
correlation between the indifference of one group and 
the interests of alTother. 

He would however expect more alienation to exist 
between the interests of some high school pupils and 
the positive dislikes of others than between the inter- 
est of some high school pupils and the neutral attitude 
of others. There is nothing in the nature of children 
or in education which creates a tendency of likes of 
some to correlate with the dislikes of others more 
closely than with a neutral attitude. Just the oppo- 
site would be more likely. This is exactly what 
we get. 

There is a negative correlation of significant size 
between likes and diaUkes (—.377); there is none 
between likes and neutral attitudes. This inner 
consistency of the data ai^es potently for the validity 
of the data. If likes and dislikes had correlated posi- 
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tively instead of negatively it would cast a severe 
criticism against the quality of the data. They 
correlate negatively. 

The correlation between neutral attitudes and dis- 
likes was +.510. This shows that there is some 
positive relationship between these two series of 
ratings. What some rate as of indiSerent value, 
others tend to rate as disliked. A common sense 
interpretation of this might be that in expressing 
opinions, indifference of some predicts dislike of others. 
Perhaps timid souls mark as indifferent what more 
valiant spirits rate as disUked.' 

One other correlation further leads us to maintain 
that our data were honestly given and represent the 
facts — the correlation between dislike and great dis- 
like, i.e., enough to advise dropping from the course, 
is +.67. This can only be interpreted if we accept 
the validity of our data. Surely there ought to be a 
high positive correlation between a tendency to dislike 
and a tendency to dislike a great deal. This relation- 
ship ought to be greatest of any in^the data. It^is 
the greatest relationship. 

Every correlation we have fits in with a notion that 
the pupils gave a fairly true report of their attitude 

' If one wished to make a general obaerratiOD of human 
oature concerning the reporta of people it would be : indifference 
is more cloeelf related to dislike than to positive like or interest 
as we use the tenn. 
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about books read. // tk^r ratings had been false or 
overcardess no such consistency of rekUionship wotUd 
have been at ali probable. 

The data upoii which the lists are constructed have 
a good reliability for t^o reasons. 

(a) The quantity of the data is too great to admit 
of a distorted picture due to inadequate sampling. 

(&) The inner consistency of the relationship 
between 

Interest and a neutral attitude (= —.184) 
Interest and dislike ( = — .377) 

Neutral attitude and dislike (= +.510) 
Dislike and great dislike (= +.670) 

gives us plenty of evidence to accept the qualitative 
superiority of the data. All the original data and 
subsequent computations used in this study are on 
file in the College of Education, State University of 
Iowa, Iowa City, Iowa. 
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CHAPTER III 

APPLYING THE CRITERION OF COMPRE- 
HENSION TO GEOGRAPHY TEXTS FOR 
ELEMENTARY SCHOOLS 

COHFBEHENSION VALUEa OF GeOQBAPHT TEXTS 

All the tables used in this study are put together 
at the end. It is believed that this will faciUtate the 
study of this research. 

This report, as part of the judgment of relative 
value of geographies, deals with the comparative 
difficulty which children find in reading the various 
texts understandingly. The method used in this 
study is of general use. No fundamental change in 
it is necessary were we deaUng with history or science 
or other texts. We would define the comprehension 
value of a text as the degree to which children may 
read passages of that text and answer simple questions 
on such excerpts without entering into any intricate 
consideration of the difficulties involved in compre- 
hending the reading matter. We can then arrive at a 
very adequate notion of the ' quality which we are 
comparing. With such a formulation in mind, w6 
have formed random samplings of the reading matter 
of each text book into testa and have asked simple 
60 
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questions on the basis of each sampling. We will then 
judge the value of any one book by children's responses 
to questions on its subject matter. 

There were five series of books which were candi- 
dates for selection. The series were lettered and each 
first book was called "1," and each second book was 
called "2."' The names of the five series are deleted 
for obvious reasons. Throughout this study they 
will be referred to as series A, B, C, D, and E. 

One coul<l not hold that reading difficulty or ease should alone 
decide which text to use. The other criteria of selection muet 
be given due weight. Books of reading difficulty too far above 
or below the reading ability of the pupils'^ who use the books are 
undeeirable. Quite often Buperior subject matter neceasanly 
involves slightly greater reading difficulty. For instance, a 
superficial description of the "tides" may involve easier reading 
than an adequate discussion of the ocean movements. Should 
the more difficult book be used the teacher should know where 
the hard passages are and govern herself accordingly. 

In order to ascribe to the book itself the adequacy 
or inadequacy of the responses of children to ques- 
tions asked, after the reading of any portions selected 
from it, we need to be perfectly sure of three major 
conditions that may influence the behavior, aside from 
the actual difficulty of the material in the text. We 
must know: 

1. That the samplings are a fair representation 
of the book and that the questions on the passages 
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from one book are no more difficult than the questions 
on passages form any other book. 

2. That the children who answer questions on one 
book are just as capable as those who answer questions 
on any other book. 

3. That the order in which the tests are given has 
no effect on the responses of the children who take 
them. 

The data which follow are organized to present 
results which adequately account for these three 
factors, and consequently we may attribute the values, 
as indicated through the responses of the children 
directly to the texts. 

The Faieness op the Sampling 

1. If the responses of the children to the questions 
on any one book were materially affected by chance 
inclusions of difficult passages, there would be no 
relation between the scores made by a group on the 
first halves of the tests and the scores made by the 
same group on the second halves of the tests. A group 
might then get a high percentage of the questions 
formii^ the first half of the selections correct, and a 
low percentage on the second half of those selections 
correct. On the other hand, if on the whole the books 
which are acclaimed as easy by selections forming the 
first half of the test (which are taken from the first 
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half of the text) are also acclaimed ob easy by selec- 
tions formiiig the eecoiid half of the test (which are 
taken from the second half of the text), then we can 
safely decide that the difficulty of the selections is a 
function of the book. Also such agreement would 
indicate that the general difficulty of the questions 
has not been greatly influenced by the mood or ability 
of the author of the test. Agreement between the 
average difficulty of the first half with the average 
difficulty of the second half of the test, shows that the 
author's personaUty was not a determining factor in 
this difficulty. 

It could be maintained that all of the questions 
to one book were easy, whereas all of the questions to 
another book were hard, but we do not believe that 
this is a serious possibiUty because of the method used 
in construction of the tests. The tests were, of course, 
constructed with strict impartiality — passages being 
taken from the same pages of each book, and crucial 
questions being asked from each passage, with no 
consideration except the logic of the passage, as the 
author imderstood it. 

Table I gives the ^^reement as it is. It will be noted 
that the books which are easy to read, by the verdict 
of children's responses to the first halves are also eaay 
to read by such verdict of the second halves. Those 
that are hard to read when we consider the first half 
are hard to read when we consider the last half. 
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There is only one exception, and that is D-1 and D-2. 
The first halves of these two books are easy to read, 
and the second halves are hard to read. It is impor- 
tant to note that these two exceptions are of the same 
series. Those books were written in that way. 

The correlation between values by the first half 
and values by the second half is + -32. The correlation 
of values by the first half with values by the total is 
+.76. The correlation of values by the second half 
with values by the total is +.83. Though correla- 
tions with caaes as few as ten are unallowable, when 
the object is to show relationship between qualities 
or between tests, it is a valuable means of showing 
characteristics of actual data, no matter what the 
number of cases. It would be unjustifiable to say 
that first halves of tests such as these correlated as 
a rule with second halves about .32, but it is entirely 
justifiable to say that these actual first halves do 
actually correlate .32 with these acttial second halves. 
By Brown's fonnula between the totals with totals 
obtained in the same way it goes up to .48, 

It will be profitable to note in this connection that 
each second book of a series is harder than its first, 
and that the volumes I place the series about as the 
volumes II do, except the E-1 and E-2, and that of all 
the series, it would be most likely that the difficulty of 
the first volume of the E series had a different relation 
to the other first volumes than the second volume of 
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this series had to the other second volumes, since 
Mr. X. wrote the first volume and Mr. Y. wrote the 
second, whereas in all of the other series, volumes 
I and volumes II were both written by one or by both 



The Equality oi- the Groups Used in the 
Tests 

2. Children whose responses are being evaluated to 
gain comparative estimates of the values of these 
texts are all twelve-year-olds. Since they are all of the 
same age and since there are from about 350 to about 
SOO in each group, the abiUties of the groups must be 
about equal, provided that their verbal inteUigence is 
about equal. Table 2 gives the distributions on WyUe 
Opposites test A, B, and C, given to these same 
children about one year before the Geography Bead- 
ing tests were given. These children were then 
eleven years old, and Table 2 represents the distribu- 
tions of WyUe scores for eleven-year-olds of the same 
schools that housed the twelve-year-olds who took 
the geography tests. "A" represents distributions of 
WyUe scores of eleven-year-olds January 14, 1921, 
in the same schools where the twelve-year-olds took 
the Geography Beading tests, A-1 and A-2, early in 
1922. "B" represents distributions of WyUe scores 
for eleven-year-olds January 14, 1921, in the schools 
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where the twelve-year-olds took Uie Ge<^raphy 
Reading tests, B-1 and B-2, early in 1922, etc. 
Table 3 gives these same facts for twelve-year-olds, 
January 14, 1921, by groups of schools which took the 
various Geography Reading tests. Table 3 gives the 
facts which insure us that it is not characteristic of 
any of these groups of schools to have twelve-year-olds 
which are much brighter than eleven-year-olds, and 
that therefore the relative abilities of the twelve-year- 
olds in these groups of schools will be just about what 
those same children were one year before, when they 
were eleven-year-olds. 

In Table 2 the percentage of zeros was for the 
"A" group, .050; for the "B" group, .070; for 
the "C" group, .070; for the "D" group, .067; for 
the " E" group, .058. The lines punctuating the distri- 
butions are percentile lines. The first line in each 
distribution marks ofE the lowest 10% from the other 
90%. The next line marks off the lowest 25% from 
the highest 75%. The third line marks the lowest 
50% from the highest 50%. The fourth line marks 
the lowest 75% from the highest 25%. The last line 
marks the lowest 90% from the highest 10%. The 
remarkable correspondence of the position of these 
lines in the five groups in both Table 2 and Table 3 
shows beyond any doubt that the groups of children 
are identical, so far as ability on the Wylie test is 
coDcerned. 
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NsaLioiBLEi Effect of ths Obdeb of Givino thb 

Tests 

3. The possibility that the order in which tests were 
giveQ might affect the responses of the children and 
would therefore introduce a factor in the evaluation 
which could not be justly attributed to the book itself, 
needs consideration. It has been repeatedly found 
that tests constructed as these were, are not easily 
amenable to practice. We found at Garden City that 
giving practice in this function so as to make an 
appreciable difference in the average of the groups 
took two years of special training; and Dr. Arthur 
Gates found at Scarborough, where the abiUties were 
already near the inteUigenee hmit, that practice in the 
function made no difference at all in the test. 

AH the "2'r" were given directly after the "I's," 
and still each "2" is harder than its corresponding 
" 1." This seems to indicate that the order of giving 
has no great influence on the results. A-1, B-1 and 
C-1 were given before any other test, and D-1 and E-1 
were given after one of the other three, both volumes 
of the series, had already been administered. In spite 
of this fact, B-1 and C-1 are pronotmced the two 
easiest, and A-1, the hardest to read. This would 
seem to indicate that the practice gained from previous 
experience in the test had no great influence on the 
results. 
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Some schools were chosen in which to change the 
order of the giving of the tests. In considering these 
results we must bear in mind that the determination 
for a sii^le school* is very unreUable, and if the 
general tendency of the results is like the results (rf the 
total, in spite of the change in order of administration, 
our conclusions will be verified. In No. 1 School 
the order of giving was E-1, D-1, C-1, B-1, — with the 
following results : 

Fkb Cxnt or CkiBBBcr 
Text Rbsfonsxb 

B-1 .76 

E-1 .67 

C-1 .66 

D-1 .63 

There were seventeen 12-year-old8, 

In School No. 2 the order of giving was E-1, E-2, 
D-2, C-2,— with the following results: 

Per Cent or Cobbxct 
Text KBBPONBEa 

E-2 .75 

E-1 .70 

D-2 .69 

C-2 .67 

There were one hundred and fourteen 12-year-olds. 

' The data used here were gatbered in a middle western 



by Google 



TEXTBOOK SELECTION 75 

In School No. 3 the order of giving was D-1, E-1, 
C-2, A-2, with the following results : 

Feb Cent or Cobbect 
txst Responses 

E-1 .81 

D-1 .76 

A-2 .67 

C-2 .66 

There were forty-aix 12-year-old8. 

In School No. 4, the order of giving was D-1, D-2, 
E-1, C-2, — with the following results: 

Pbb Cent of CoBBEcr 
Text Resfonsbs 

E-l .67 

D-1 .65 

D-2 .57 

C-2 .54 

There were sixty-eight 12-year-oldfl. 

In School No. 5, the order of giving was E-1, E-2, 
C-2, D-2, with the following results: 





Pm Cbnt of Coi 


TXZT 




E-I 


.59 


D-2 


.68 


E-2 


.57 


C-2 


.51 



There were fifteen 12-year-old8. 
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In School No. 6, the order of giving was D-2j E-1, 
£-2, C-2,with the following results: 

Feb Cent or Corbect 
Text Rebfonses 

E-2 .39 

E-I .37 

C-2 .27 

D-2 .22 

There were nine 12-year-olds. 

Certain noticeable features deny "the order of 
giving" any part in the results. Certainly the test 
which is given last is not always the easiest. The only 
case in which it is easiest is in School No. 1, and there 
it is because B-1 was given last, and B-1 is by our total 
result the easiest. Also there is no case in which the 
test which is given first is demonstrated to be hardest. 
E-1 is always about as hard as D-1, as it is in our totals, 
no matter what the order of giving. As a matter of 
fact, E-1 was easier for the children of School No. 3 
School No. 4 and School No. 5 than D-1, even though 
E-1 was given first in No. 4 and No. 5, and D-1 was 
given first in School No. 3. A-2 bears the position in 
difficulty in School No. 3 that it has in our totals, even 
though it was given last. C-2 is the hardest for the 
children at No. 1, No. 3, No. 4 and No. 5, and next to 
the hardest for the children of No. 6, even though it 
was given last, or next to the last in each, — that is to 
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Bay, C-2 which is hardest by our summaries ^hen 
practice in only one test (C-1) had been given, is 
also hardest in these special schools when practice in 
two or three tests had been given before the adminis- 
traction of C-2. E-2 is easier than C-2 and D-2 in No. 
2 and No. 6 even though it is given before C-2 
and D-2, and is about as hard as D-2 and easier than 
C-2 in School No. 5, even though it is given before C-2 
and D-2. 

This analysis of the schools where a special order of 
giving Mas instituted seems to indicate that the data 
are independent of the order in which the tests were 



How Reading DifficuIiTt op the Texts Wab 
Obtained 

The reports came to us from each school, listing 
the number of correct responses, the number of 
incorrect responses, and the number not attempted, 
for each question. We were therefore able to deter- 
mine the percentage that the correct responses were 
of the total attempts. As indicated above, we threw 
out of consideration any school in which the "not 
attempted" was larger for any questions at the end 
than for questions at the beginning of the test, since 
this indicated that insufficient time had been given 
to adequately measure the responses of the children 
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to the material itself. This "percentage correct" 
for each book constitutes the third column in Table 1 . 
We have here a picture of the relative ease with which 
twelve-year-olds read the ten volumes in question. 
Table 4 gives these percent^es by series. 

As it is somewhat difficult to interpret the mesmng 
of these differences, we have translated them into s 
more readily understood medium. Table 5 shows 
the deviations in terms of Standard Deviation and 
the average age necessary to read the texts. The 

percentage of correctness was translated into oTT' 

deviation in terms of the variability of twelve-year- 
olds. This deviation was translated into a scale 
rai^ng from five standard deviations below the mean 
to five standard deviations above the mean, — calling 
the former zero, and the latter 100. This places the 
average at 50. We have values now for each book 
which correspond to McCall's T-acore on the Thom- 
dike-McCall Reading Scale. In Table 5, B-1 is ea 
hard as 43 on the Thorndike-McCall Reading Scale. 
C-1 is as hard as 45 on the Thorndike-McCall Reading 
Scale, etc. 

Whereas these terms are indispensable statistically 
to arrive at a final simplified report, they are in them- 
selves so intricate that it is very important to present 
our relative values in a manner more easily undei^ 
stood. Once arrived at a T-score, it is, a mmplB 
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matter to use the Thomdike-McCall age normB to 
find what age is necesBary to comprefaentj any one 
book. B-1 is as hard as 43 on the reading scale, and 
the average age of the T-score of 43 is, according to the 
McCall norms, 130 months. We can therefore, say 
that children must be this old in order to undei^tand 
B-1. So with the other reading ages in Table 5. 
It is also of value to us to know which is the lowest 
grade able to read each book. These values are 
derived through the grade norms published by McCall 
for T-score of his test. We now have before us in 
Table 5 the average age necessary to read each book 
and the average grade necessary to read each book. 

Sblbctinq Texts to Meet the Needs of Particu- 
lar Grades and Students 

One other immediate advantage, other than the 
selection of texts, should be parenthetically answered 
here. By test the mental age of grades used in this 
study was determined. Thus we could see which 
grades in any one school are like a fourth grade normally 
would be; which grades are like a fifth grade normally 
would be, etc. In this way individual needs of each 
grade of this city's schools can be seen at a glance. 
As these scores are on a verbal intelligence test, the 
differences are in exactly the same ability upon which 
the relations between texts are based. The eorrela- 
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tioD between the Tbomdike-McCall and the Wylie 
is high. 

A-1 mi^bt very well be the best book in every other 
respect than comprebension. In fact, it« very values 
may be associated with a necessary sacrifice in terms 
of comprehension. In that case it m^bt be very 
advisable to use A-1 in all schools where tbey are 
able to comprehend it, and to use the book which 
may not be as valuable, but easier to read, only in 
those schools where it is necessary to sacrifice the 
other values in favor of the sine qua non, comprehen- 
sion. The statement that A-1 is better in other ways 
is of course entirely hypotheticid. 

Thb Vaeiabilitt of Reamno Difficulty WrrmN 
A Single Text 
There is a further characteristic of geography text 
books which our data will allow us to analyze. The 
difficulty of a book as a book is one major considera- 
tion in the^treatment of comprehension value, but 
the degree to which the difficulty varies from very 
easy to very hard is a very important supplementary 
consideration. B-1 is easier than any of the other 
volumes-l to read. However, if this averse ease of 
comprehension were the result of the product moment 
of very easy and very bard passages, it would never- 
theless be an undesirable text. We want a book the 
difficulty of which is reasonably constant throughout. 
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Tables 6 and 7 preaent the T-scores and reading 
ages of each question. The reading Age is to be 
interpreted as the average age necessary to read the 
passage and answer the questions as per the conditions 
of our test. Some of the disparity in ^es is of course 
due to the sampling of pass^es and the chance diffi- 
culty of the questions, but the range of difficulty of 
the text book is certainly pictured in a rough manner 
by the range in difficulties of these passages. 

It is especially of note that the difficulty of the 
sentences in D~l and D-2 marches progressively from 
easy to hard, which fact is already noted in connection 
with Table 1, where the first halves of these two books 
were shown to be easier than the second halves. The 
questions are in serial order, and samplings are from 
the beginning to the end of the book. Whether or not 
this feature ia desirable is dependent upon the progress 
which children can make in ability to read. The 
difference in difficulty between the end and beginning 
of the D's is not justified by the progress which 
children make in the time covered by the books. 

Table 8 gives the standard deviation of reading 
ages of the questions of each book and the average 
readily age of the book. It is because of dropping of 
decimals that the average age of questiopB per book 
does not check with the age necessary to read the 
book, — computed directly. However, the order of the 
books is of course the same. A consideration of these 
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measures of variability of the difficulty of passages 
from the various texts expressed in terms of ^e 
shows that as a tendency, the first and second volumes 
of a series have variabilities very much alike. Note 
that B-2 and B-1 are tc^ther; note that D-1 and D-2 
are together; note that A-1 and A-2 are very dose 
together; note that E-1 and E-2 are fairly close together. 
C-2 and C-1 are very different in variability, but all the 
other series have Standard Deviations of the volumes-l 
which show a likeness to the Standard Deviations of 
volumes-2. The highest Standard Deviations are for 
A-2, C-2 and A-1. These three volumes are among 
the four most difficult to read (Table 5) . The lowest 
standard deviations are those for C-1, E-2, B-2andB-l. 
All of these, except B-2 are among the five books which 
are easiest to read (Table 5). 

There is then, lucidly, an association of comprehen- 
sion virtues in these texts. Those that are hard to 
read are also spotty in their difficulty and those that 
are easy to read have an even tenor of difficulty. 

The Success of the Texts in Reachino Thbib 
Readinq Ptblic 

Table 9 gives the frequency of each reading age 
of all questions on all volumes-l. An analysis of 
this frequency distribution of average age necessary 
to answer each of 104 questions, after reading the 
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paes^^ selected from fire geography text books 
meant for beginners in geography in grades 4 and 5, 
will allow us to judge how far the authors of the texts 
have reached their public. Their public is, of course, 
the children themselves, — not what teachers think of 
those children. Although the sales today may be 
largely based on what superintendents and teachers 
think children of the fourth and fifth grade can read, 
rather than what they actually do read, it is just such 
data as t^ese which will change that emphasis and 
allow ua to judge text books ufton a basis of their real 
ability to get across to the children that they are 
meant for. The average of the average age necessary 
to read these questions is 135.79 months. With a 
standard deviation of about a year and a quarter, 
about five-sixths of the material is too hard for children 
in grades 4 to understand. Since one standard 
deviation below the mean of the distribution, repre- 
sented in Table 9, is at about 120 mouths, if we count 
the children in grades 4 as nearly all younger than 120 
months in reading ability, five-sixths of the passages 
of the texts given them are too hard for the group. 
Volumes-2 ore a little more adequate to the task 
which they pretend to perform. The average of the 
average ^e necessary to read the 111 questions on 
volumes-2 is 143.50 months. The standard devia- 
tion of reading age of queetioos is again about a year 
and a quarter. This distribution of reading ages by 
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queatioQB on Tolumes-2 will be found in Table 9. If 
TolumeB-2 are meant primarily for grades 6 and 7, 
only about one-half can be understood by the children 
of the grade in which they are first introduced, (Grade 
6.) Since the average age necessary is about twelve 
years, if we count most sixth^raders as not more than 
twelve years old in reading ability they are then able 
to understand only one-half of the matter presented 
to them in all geography texts meant for the grade. 
This one-half is not the first halves of the book as is 
apparent from Table 7. 

Individoal Difpbbences in Needs of Various 
Grades 

On January 14, 1821, the 5-A grades of the schools 
used in this research, about to enter on their 6th 
grade work had average scores on the Wylie A, B and 
ranging from 55.15 to 19.69. Interpreted, thif 
means they range from an average age of 12 years 
6 months to 8 years 6 months, in the ability measured 
by the Wylie teat. They differ in the ability of work 
which they can do as measured by the Wylie test 
from average 7th grade capacity at beginiiii^ of B 
work to average 4th grade capacity (at point of 
promotion from B to A). 

Associating these facts with those of Table 5, it is 
apparent that some 6th grades are able to use any of 
the texts, whereas others need to choose the very 
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easiest and then still will have a book which is too 
difficult. The best 6ih grades cau just read the hardest 
volume-2 and not all can read the easiest. This is a 
very valuable practical consideration since we may be 
able to determine as' sui^ested above that a certain 
book excels the others in all quaUties save ease of 
reading. Then we must know in which grades the 
difficulty of comprehension makes no difference in 
order to benefit where we can by the excellence of the 
book in other qualities. 

In order to find which 6th grades (and for volumes-l 
which 4th grades) are able to use a difficult book, 
consult the mental age ratings of each grade. 

Thb Tbxt ALBB&nT IN Use Had Neguoible 
Advantaqe 
In oonclumon a word regarding the ge<%raphy now 
in use in the lower grades is necessary to assure us that 
training in it has not prejudiced our results in favor 
of one of the books. "B," First Book, Part One has 
been used the last three years. Since the first 90 
pf^es of the 256 (exclusive of indices) of our B-1 is 
this same Part One, we must be sure that questions 
on this portion are at least as hard as questions on the 
remainder of B-1. This portion that has been studied 
covers questions on our test I to 5 inclusive. Thoi^h 
three of these five questions were comparatively easy 
for our 12-year olds (see Table 6) still Table 1 shows 
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that the second half d B-1 (which was Dot studied) 
waa easier than the first half (part of which was 
studied). We feel therefore that no important advan- 
.t^e was given the "B" series. 

This treatise is of comprehension value of the texts 
only. It makes no claims to ascertain any of the 
other necessary virtues of a text. When those too are 
credited through objective means we will be able to 
choose texts, not risk them. 

Tablb 1 



" 


Percentage of 


Percentage 1^ 






questions on 


queetions on 


queetions on 


Key number 
. oftort 


fintbalfof 


second ^ft-lf of 


total of test 




test answered 




correctly by a 


correctly by a 


rectlybya 




normal group 


normal group 


normal group 






of 12-year.olda 


of 12-year-olds 


B-1 


68 


77 


72 


C-1 


71 


67 


69 


D-l 


79 


54 


66 


E-1 


64 


68 


66 


B-3 


63 


64 


64 


&2 


62 


65 


64 


A-1 


62 


61 


62 


D-2 


64 


51 


68 


A-2 


B2 


59 


56 


C-2 


54 


48 


51 
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TaBL2 2, — DiSTBIBUTIONB IN WtLIB A, B AND C, 

Januabt 14, 1921 BT Gboups of Schools 

Taking A, B, C, D and E Gbographt 

RsADiNQ Tebts-Eleten-teab-oldb m 

Those Schools at That Tiue 





A 


B 


c 


D 


S 







IB 


34 


31 


41 


33 








20 








-•10» ol til. number bdow thil 


6- 10 


32 


21 


18 


36 


26 


liiHineuhdiitTibutioD. 


10- IS 


6 







10 


13 




15- 30 














ao-as 


6 




IT 


18 


23 


-*2&% ol the Dumber below tUt 


3s- so 


la 


34 


17 


20 


27 






20 




36 








85-40 


20 


18 


21 


37 


31 




10-*S 




3S 


26 


24 


83 




4&- 60 


30 


SO 








-»Thie Ime hslvo the total 


60-66 


2S 


47 


3T 


46 


56 




58-60 


38 


35 


33 


48 


47 




60- 66 












->35» of the number abarethU 


96- TO 


26 


44 


29 


40 


38 




70- 78 


17 


33 


38 


37 


24 


-*10B of «ie;nuinb« ebore thia 


76- 80 












line in e»h diitribution. 


80- 86 


13 


13 


11 


13 


IT 




S5-90 


S 


6 




10 


7 




SO- 06 










7 




96-100 


4 












100-106 


2 









3 


















110-115 















115-120 


1 


2 




1 


3 




120-125 










1 






344 


tse 


MS 


613 


678 
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Table 3. — DisTRmnnONs in Wtlib A, B, and C, 

Januabt 14, 1921, BT Gboufs of Schooi^ 

Takinq a, B, C, D and Q Gboobapbt 

Rbadino Tests — ^Twelve-teab-olds in 

Tbose Schools at Tbat Tucb 





A 


B 


C 


D 


B 







IS 


26 


17 


28 


3B 




0- 6 


H 












6- 10 


11 




18 




12 




10- IS 






S 


13 


8 




16- 20 


e 


7 






14 




20- 2S 














26- 30 




11 


12 


17 


19 




30- 3S 




22 




23 


28 




SS- 40 








2S 


38 




40- *6 


ae 


23 


12 


34 


33 




«s-eo 


10 


19 


24 


37 


26 




Bo-ea 














05-M 


37 


28 


34 


43 


40 




<KK 05 


3S 


39 


87 


6S 


60 




6S-T0 






3E 








Ta-7S 


31 


23 


38 


33 


34 




7»-80 


16 


21 


36 


31 


86 




80- 8S 






3S 




23 




86-90 


18 




18 


IB 


10 




00- OS 


e 




13 


18 








8 






3 






100-106 














lOS-110 









8 




















116-180 














130-12S 








a 






iaS-130 














130-lSS 

















m^ 


307 


444 


612 


Z. 
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Table 4 

Pebckntaqb or Qukstionb 

7 SlIBlSS COBBXCT 

B 67.7 

E 64.8 

D 62.1 

C 59.4 

A 58.6 
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Table 5. — Distinctions in Compbbhension Value 
Translated into Aoe and Grade 



Tsrt 


ofoorreot 


normal dis- 
tribution ol 
diffiouhy 


Tiosle 
like ThorD- 
dikB-Mo- 
C^tMd' 


Mry to oom- 
piehead 


Avena« 

end. 
nMHBrr 


B-1 
C-1 
D-1 
E-l 
B-3 
E^ 
A-I 
D-2 
A-2 
C-2 




09 

66 

04 
U 

B8 
S6 


: 


58 
60 
41 

SO 
86 

20 

a 

02 


44 
46 

4S 

4e 

4B 
49 


3 


136 

las 

189 
13S 

144 

IS 




48 

GO 

74 
74 

00 
09 










WholB 








Note wlumni above for 


B 

D 
C 
A 




68 

aa 


: 


47 
89 
31 
23 
23 


4G 

46 
47 




136 

las 

141 
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S 
6 
E 
S 
S 


89 
82 
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Table 6. — T-scobbs and Reading Aqbs bt Ques- 
tions IN Tests on Voltjues-I 







s 


, 


« 




. 1. 








S 


NumbBT of 
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1 


h 


f- 


h 


1 h 


E- 


fi 


1 


|s 




B-1 


C-1 


D-I 


B'l 


A-1 


J 


40.1 


131 


48.2 


14G 


39 


3 119 


34 


4 


106 


43.^ 


139 








46.7 


140 


SO 






2 


13S 


40.8 


123 




K.3 


130 


3B.2 


117 


33 


G 103 


52 


8 


167 


50 




151 




M.7 




43.3 
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30 






a 


























4 








143 




46.0 


13S 


43.6 


132 


43 
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37 


7 


115 






134 




40.8 


148 


51.0 


163 


38 
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13S 




46.1 




42.6 










8 




48 








6fl.7 
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40.6 
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43 
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4S 


7 


140 
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43.3 


128 


40. S 


133 


44 
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43 
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129 


13- 
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45 
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46 
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148 


43 
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148 


4K.3 
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48 




40 
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117 
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64 
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46 
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148 
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TaBLB 7. — ^T-SCOBEB AND RbADINQ AoB8 BT QUES- 
TIONS IN Tests on Voltjmbjs-2 
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15B 


40 




123 


41 




136 


37 


3 


114 


10 


42 




128 


43.0 


187 


43 




138 








48 


7 


140 


11 


44 






40, B 


122 


17 




143 


43 




137 




6 




13 


49 




140 


51. 8 


154 


51 




lfi4 


43 




137 


B7 


7 


171 


13 


53 




lES 


(3.3 










35 




110 


B7 


1 


189 


H 








44. S 


las 


S3 




1B8 


41 




126 


SO 


8 


163 


IB 


S4 




163 


43.9 


133 


53 




160 


41 




12B 


47 


2 


143 




88 




117 


43,0 


133 
















6 












45. a 


187 


51 




182 






148 




8 




18 

la 


4S 
46 




133 
140 


47.7 


US 


59 
48 




177 


61 




153 


51 


B 
7 


164 
118 


20 












4S 




138 


i( 






BO 







SI 


4S 




188 






43 




130 


IB 




138 


53 


9 


181 


aa 












43 




133 






143 


53 


3 


159 


2) 












El 




1B3 


42 










34 












BO 




179 


48 




144 






as 


















4S 




136 






20 


















40 




12S 
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TEXTBOOK SEUICTION 93 

Tablb 8. — Stahdabd Deviations and Avebagbs 07 

Reading Ages of the Questions bt Book 

IN Ohdeb of Size of Standard Deitiations 



Book 


S.D. 


M 


C-1 


9.45 


134.68 


E-2 


11.48 


139.11 


B-2 


11.79 


138.86 


B-1 


14.55 


130.95 


E-1 


14.64 


136.37 


D-2 


15.56 


143.79 


D-1 


16.79 


136.75 


A-l 


17.32 


140.36 


C-2 


17.66 


148.14 


A-2 


18.79 


146.08 
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Table 9. — DiBTHiBTrTroN op Reading Age Neces- 
8ABT TO Read Questionb on Voluheb-I and 

VOLnME8-2 



Reading age 


Volmnes-l 


Volume8-2 


100-105 


1 




106-110 


2 




110-115 


6 


3 


115-120 


9 


4 


120-125 


7 


4 


.125-130 


11 


13 


130-135 


15 


8 


135-140 


15 


19 


140-145 


12 


15 


145-150 


E9 


5 


150-155 


i* 


11 


155-160 


h 


9 


160-165 


4 


5 


165-170 


4 


8 


170-175 




3 


175-180 




4 


180-185 


1 






VoIumes-1 


VoImne8-2 


N 


104 


111 


M 


136,20 


143.95 


S.D. 


15.25 


16.05 
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